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ABSTRACT 


The importance of information superiority has been emphasized as a critical 
capability that future joint forces must be able to achieve. No longer simply a future 
concept, it is being officially defined and incorporated in doctrinal publications like Joint 
Publication 3-13, Information Operations. Unfortunately, our ability to effectively 
measure its contribution relative to other battlefield systems remains limited. This 
research focuses on exploring the limits of the contributions that information superiority 
can make, examining the sensitivity of information superiority to varying information 
quality and comparing those contributions with other contributing factors to battlefield 
results. Furthermore, an effort is made to identify some of the risks associated with using 
information superiority as a force multiplier. A simple decision model was developed 
based on the concepts of a two-person zero sum game to explore these questions. In the 
model, one side is provided varying degrees of an information advantage, while also 
varying degrees of information quality to the information advantage. Additionally, a 
variety of scenarios were considered involving varied levels of opposing side force 
levels. Experimental design techniques were employed to efficiently explore the model 
output space while allowing for sufficient replications of the model at each design point 


in order to provide a sufficient data set for analysis. 
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EXECUTIVE SUMMARY 


Information superiority is a leading concept driving joint future force 
development. Proponents view it as a force multiplier; given forces of equal size and 
ability, the one that possesses information superiority can achieve superior results to that 
of the other. Research suggests that this is, in fact, the case. Yet, what are the risks 
associated with units relying on information superiority? How can we measure the 
degree of superiority that an information advantage provides? How much is enough? In 
a world constrained by budgets, these are important questions to be answered so that a 
proper balance can be made between equipment meant to destroy our adversaries and 
equipment that facilitates information superiority. It has been aptly pointed out by 
General Howell M. Estes III, United States Air Force, a former commander of Space 
Command that, “...you can’t take out an enemy tank with just information.”! 

Unfortunately, attacking this type of problem has proven difficult for the 
Operations Research (OR) community. Alan Washburn, Professor Emeritus at the 
Naval Postgraduate School, views the situation as somewhat dire. He says, 

There is a crisis for military OR, centered on the role of information on the 

battlefield. It is clear to military professionals that information is becoming 


increasingly important, but unfortunately the OR profession’s ability to measure 
its contribution is still primitive.” 


This present work will not attempt to make a breakthrough in these “primitive” measures, 
but, instead, will try to creatively apply methods on hand to continue to explore the value 


of information superiority in addressing the following research questions: 


e How do varying degrees of information superiority affect battlefield 
outcomes? 
e How sensitive are these outcomes to the quality of information used to 


obtain information superiority? 


! Alan R. Washburn, “Bits, Bangs, or Bucks? The Coming Information Crisis,” PHALANX, Vol. 34, 
No. 3 (Part I), 2001, p. 6. 


2 Ibid., p. 6. 


XV 


e What is the cost savings in terms of level of forces that information 
superiority provides (i.e., given a certain level of success achieved through 
information superiority, if information superiority is taken away, what 
increase in force size is required to achieve similar results)? 


The title of this paper is meant to convey the link that exists between information 
superiority and a decision maker. Information superiority is defined as “the operational 
advantage gained by the ability to collect, process, and disseminate an uninterrupted flow 
of information while exploiting or denying an adversary’s ability to do the same.’ 
However, this advantage is turned into improved battlefield success through the 
decision-making process. Therefore, in this study we employ a simple decision model 
based largely on the concepts of game theory, specifically a two-person zero sum 
(TPZS) game. 

TPSZ games are an excellent abstraction of military conflict since they involve 
two opposing sides, each of whom must decide between an array of strategies in order to 
achieve the most desirable result. Warfare, in its most basic form, can be thought of as a 
series of decisions that are executed in the form of strategy and tactics between opposing 
forces, where each side desires a higher payoff. In the case of the TPZS game where 
both sides have no information as to how the other will behave, the optimal action for 
each side is to play their respective strategies with the proportion that guarantees the 
maximum expected value, regardless of the strategy the adversary selects. This is known 
as the optimal mixed strategy. In this situation, the opposing sides have information 
parity—neither holding an advantage over the other. This situation serves as the baseline 
of comparison to situations where one side holds an information advantage. 

In order to incorporate the information advantage concept, the TPZS game used in 
this study is a larger decision model that maintains the principles of the TPZS game. In 
this larger decision model, one side is consistently provided varying degrees of 
information superiority to explore how the increased information advantage impacted the 
expected payoff. The difference between this expected payoff and the baseline expected 


payoff is measured as the value of information superiority for the given circumstances of 


3 Joint Chiefs of Staff, Joint Publication 3-13, “Information Operations,” Washington D.C., February 
2006. p. I-5. 
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a specific game. Additionally, the quality of the information is varied, incorporating 


model parameters for timeliness and accuracy of information. 
Experimental results have suggested the following: 


e The value of information superiority is not uniform, but is strongly 
influenced by force ratio and force size. 


e Information superiority can be increased to a point that it returns little or 
no value. 
® Decreasing information quality degrades the value of information 


superiority uniformly, with no single drop-off point. 


e While there are risks to relying on information superiority in terms of 
potential decreases in battlefield performance, there is a much greater 
potential for increased battlefield performance. 


It appears that nonlinear relationships exist between the value of information 
superiority and both force ratio and force size. There is an initial minimal force 
requirement before information has much value. Additionally, one force size reaches a 


certain point, the value of information superiority begins to decline sharply. 


This study is meant to continue to contribute to a critical area of operations 
research where much work remains to be done. While not revolutionizing the way we 
measure the contribution of information to war fighting, it demonstrates a practical 
manner in which to explore the problem and develop useful insights. Additionally, by 
building on some of the research that has gone before, this study should help illuminate 


future areas of similar interest. 
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I. INTRODUCTION 


A. STUDY MOTIVATION AND THESIS 


Joint Vision 2020 states that, “The continued development and proliferation of 
information technologies will substantially change the conduct of military operations.”4 
As part of this change, the concept of information superiority has been identified as a key 
enabler to future U.S. success on the battlefield. This concept has now been codified in 
joint doctrine such as JP 3-13 Information Operations, where it states, “To succeed it is 
necessary for US forces to gain and maintain information superiority.”> This publication 
goes on to define information superiority as “the operational advantage gained by the 
ability to collect, process, and disseminate an uninterrupted flow of information while 
exploiting or denying an adversary’s ability to do the same.” With these things in mind, 
it is safe to say that the U.S. military is seeking to take advantage of these changes in 
information technology to achieve a sustainable operational advantage on the battlefield 
in the same way that artillery with a range of 15 km has a sustainable operational 
advantage over artillery with a range of only 10 km. Part of the promise of these changes 
is revealed in Joint Vision 2010 (JV 2010), where it is envisioned that “we should be able 
to change how we conduct the most intense joint operations.”” JV2010 goes on to posit: 
“Instead of relying on massed forces and sequential operations, we will achieve massed 
effects in other ways’’’, thus avoiding “risky massing of people and equipment.”? But 
what are the risks associated with smaller, more dispersed units relying on information 


superiority? And how can we measure what the contribution of information superiority 


4 Joint Chiefs of Staff, Joint Vision 2020, Joint Staff Pentagon, Washington, D.C. 
http://www.dtic.mil/jointvision/jvpub2.htm, accessed December 2007. pp. 3-4 





5 Joint Chiefs of Staff, Joint Publication 3-1 3, “Information Operations,” Washington D.C., 
February 2006. p.ix. 


6 Thid., p. 1-5. 


7 Joint Chiefs of Staff, Joint Vision 2010, Joint Staff Pentagon, Washington, D.C. 
http://www.dtic.mil/jv2010/jv2010.pdf, accessed December 2007. p. 17. 


8 Ibid., p. 17. 
9 Ibid., p. 17. 





will be to the “massed effects” that we desire? It has been aptly pointed out by 
General Howell M. Estes III, United States Air Force, a former commander of 
Space Command, that “You can’t take out an enemy tank with just information. We need 
to strike a balance between ‘shooters’ and ‘information systems’ if we’re going to be 
successful in the future.” !9 

Alan Washburn, Professor Emeritus at the Naval Postgraduate School, has 
pointed out that the operations research community is not well positioned to address the 
issues related to the balance between “shooter” and “information systems” because 
effective ways for measuring the contribution of information on the battlefield, in 
general, are lacking. He says, 

There is a crisis for military OR, centered on the role of information on the 

battlefield. It is clear to military professionals that information is 


becoming increasingly important, but unfortunately the OR profession’s 
ability to measure its contribution is still primitive.!! 


Professor Washburn remains pessimistic about our ability to attack the problem, but 
urges creative use of the tools at hand. His article provides a useful summary of available 
tools that presents their relative weaknesses and limitations. 

Thus, this study takes aim at exploring what is clearly an important research area 


by creatively applying the tools at hand to look for insights into the following questions: 


e How do varying degrees of information superiority affect battlefield 
outcomes? 
e How sensitive are these outcomes to the quality of information used to 


obtain information superiority? 


e What is the cost savings in terms of level of forces that information 
superiority provides (i.e., given a certain level of success achieved through 
information superiority, if information superiority is taken away, what 
increase in force size is required to achieve similar results)? 


The first question addresses issues related to how much information superiority is 


enough. Of course, we expect that an increase in information superiority will result in 


10 Alan R. Washburn, “Bits, Bangs, or Bucks? The Coming Information Crisis,” PHALANX, Vol. 34, 
No. 3 (Part I), 2001. p. 6. 


11 Ibid., p. 6. 


better battlefield outcomes, but is the relationship linear and how steep is the slope? The 
second question examines the sensitivity of the effects of information superiority when 
the information being used to develop information superiority has varying degrees of 
quality. The quality of information is defined as the information’s accuracy, relevance, 
timeliness, usability, completeness, brevity, and security.!2 For this study, we will vary 
information quality by varying the accuracy and timeliness characteristics and holding 
the remaining factors at steady positive levels. There is research that has examined the 
effects of information quality and information superiority,!? but not much that has 
addressed both simultaneously, as this work is attempting to do. Finally, the third 
question is looking for a comparative analysis between force size and the level of 
information superiority possessed. An example comparison would be, given a force of 
specific size with a certain level of information superiority that can achieve some level of 
success in a given scenario, if we create another force without any information 
superiority, how large will it have to be to achieve an equal level of success or chance of 


success? This is the aim of this present work. 


B. THE GENERAL VALUE OF INFORMATION 


Information superiority provides the joint force a competitive advantage 
only when it is effectively translated into superior knowledge and 
decisions. The joint force must be able to take advantage of superior 
information converted to superior knowledge to achieve ‘decision 
superiority’—better decisions arrived at and implemented faster than an 
opponent can react... .!4 


Warfare, in its most basic form, can be thought of as a series of decisions that are 
executed in the form of strategy and tactics between opposing forces. Viewed in this 


light, it is clear that information serves a decision maker and its value is related to the 





12 Joint Chiefs of Staff, Joint Publication 3-13, “Information Operations,” Washington D.C., 
February 2006. p. I-3. 


13 Gary A. McCintosh, ” Information Superiority and Game Theory: The Value of Varying Levels of 
Information,” Master’s Thesis, Naval Postgraduate School, Monterey, CA, March 2002, is a specific 
expansion on Bracken and Darilek’s (1998) work. Another interesting study on information quality is 
McGunnigle, J. Jr., “An Exploratory Analysis of the Military Value of Information and Force,” Master’s 
Thesis, Naval Postgraduate School, Monterey, CA, December 1999. 


14 Joint Chiefs of Staff, Joint Vision 2020, Joint Staff Pentagon, Washington, D.C., 
http://www.dtic.mil/jointvision/jvpub2.htm, accessed December 2007. p. 11. 
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degree in which it can potentially improve a certain decision with the ultimate aim of 
improving action on the battlefield. As Washburn points out, “Information is of no value 
unless there is an uncertain decision maker.” !5 From this, we conclude that information 
superiority is not an end in and of itself. What is really desired is decision superiority; 
better and faster decisions relative to that of the adversary. 

These ideas were captured very succinctly back in the 1970s by Colonel John 
Boyd, United States Air Force. Boyd’s decision-making model, known as the Observe, 
Orient, Decide, and Act (OODA) Loop, was developed as a model for air-to-air combat, 
but is now generally applied to all forms of decision making. The first two steps 
(Observe and Orient) can be considered as the information-processing steps, where 
information is collected, organized, and interpolated. The second two steps (Decide and 
Act) can be considered as the steps where the information is utilized and turned into 
actions on the battlefield. It is a very useful model for visualizing the basic dynamic of a 
decision maker. Of course, decision makers do not always make good decisions. In fact, 
sometimes they make very poor decisions. Assuming we have an individual in a position 
to make a decision, what are some potential sources of error or improvements that enter 
into the decision-making process that could effect the actual decision made? To answer 
this, we look to a more complex model called the Dynamic Model of Situated Cognition 
(DMSC).!6 

The DMSC is a model for describing the development of a decision maker’s 
situational awareness. The result of the information-processing steps (Observe and 
Orient) is what is generally known as situational awareness,!7 which is simply a term to 


describe an individual’s comprehension of the events going on around them. It has no 


15 Alan R. Washburn, “Bits, Bangs, or Bucks? The Coming Information Crisis,” PHALANX, Vol. 34, 
No. 3 (Part I), 2001. p. 6. 


16 Nita L. Miller and Lawrence G. Shattuck, “A Process Model of Situated Cognition in Military 
Command and Control,” Proceedings of the 2004 Command and Control Research and Technology 
Symposium, San Diego, CA. 


17 Definitions for situational awareness are numerous. The Marine Corps definition may be the most 
concise and can be found in MCDP 2, Intelligence: “...a keen understanding of the essential factors which 
make each condition unique—rather than on preconceived schemes or techniques.”, but this hardly 
qualifies it as the most used or understood. The journal Human Factors, Volume 37, Number 1, March 
1995, is completely dedicated to the discussion of situational awareness and is a good place to start if you 
are interested in that larger discussion. 
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specific geographic boundaries, but its significance comes back to the comprehension of 
relevant information that could potentially improve a decision that needs to be made at a 
specific time and place. Thus, we can say that information superiority facilitates 
situational awareness, which, in turn, facilitates decision making. Figure 1 is a graphic 


description of the DMSC. 
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Figure 1. Dynamic Model of Situated Cognition!® 


By way of a brief description, Oval 1 contains all the information in the 
environment—the ground truth, if you will. Oval 2 is information collected from the ground 
truth. But Oval 2 is more than simply a slice of Oval 1; Oval 2 also contains distortions and 
misrepresentations of Oval 1. Finally, Oval 3’s relationship to Oval 2 is similar to the one 
that Oval 2 has to Oval 1, in that additional errors can be promulgated as information is 
displayed on available command and control systems. These errors can be the result of 


enemy actions as well as friendly mistakes and miscommunications. At this point, the 


18 Nita L. Miller and Lawrence G. Shattuck, “A Process Model of Situated Cognition in Military 
Command and Control”, Proceedings of the 2004 Command and Control Research and Technology 
Symposium, San Diego, CA. p. 3. 
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decision maker has not even come into contact with the information. Oval 4 contains the 
data perceived by the decision maker, which has been filtered, for better or worse, by the 
perception lens (Lens A). Oval 4 is akin to the Observe step in the OODA Loop. The 
important nuance here is that errors may already exist in what the decision maker is 
observing since, in this context, the observations are indirect. Next, the information flows 
from Oval 4 to Oval 5 through the decision maker’s interpretation lens; also akin to the 
Orient step of the OODA Loop. Oval 6 could be summarized as the resultant situational 
awareness of the decision maker once the information processing gets through the 
extrapolation lens (Lens C). 

This DMSC model does an excellent job of explicitly and implicitly displaying 
the numerous factors that can improve or degrade a decision maker’s situational 
awareness, whether they are related to command and control, Intelligence, Surveillance 
and Reconnaissance (ISR) limitations, redundant reporting, enemy deception, human 
error, or many more factors. In doing this, it also highlights the complexity associated 
with any attempt to mathematically model and explore the contribution of information 
superiority. It is also important to note that achieving information superiority is not an 
objective goal, but one relative to the state of your adversary. Vice Admiral H. Denby 
Starling I, United States Navy, commander of Naval Network Warfare Command, 
described it succinctly this way: 

ISR (collecting information about enemy and terrain) 
+ 
C4 (keeping track of friendly info) 
+ 

Information Operations (Attacking enemy’s information while protecting 

friendly info) 


INFORMATION SUPERIORITY !? 


19 Secretary of the Navy Guest Lecture, Naval Postgraduate School, Monterey, CA, | April 2008, 
Admiral Denby Stargill, United States Navy (personally attended by the author). This reference is a 
depiction of a concept presented on one of Adm. Stargill’s slides but is not meant as a word for quotation. 
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Thus, this formulation does not imply that the fog of war can be eliminated, but 
that a gap can be achieved between our adversary’s ability to use information and our 
own. The result should be increased situational awareness for us and a decreased 
situational awareness for them, which, in turn, should result in better decisions and a 


better outcome from our perspective. 


C. MODELING APPROACH AND SCOPE 


The single most important factor in model development to explore these concepts 
was having a model that involved a decision maker in an adversarial situation. Following 
this requirement, there was also a preference for a closed loop model in order to take 
advantage of modern computing power for conducting numerous replications of a 
stochastic process. Additionally, simplicity was an important factor in model 
development, with the idea of examining relatively simple decision-making situations at a 
high level of abstraction as a starting point for exploration. This led to the development 
of a model largely based on game theory and the concepts behind two-person zero sum 
(TPZS) games.29 The idea behind a TPZS game is that any benefit to one player results 
in the direct detriment of the other player. The applicability to conventional warfare is 
immediately apparent, where often two sides are competing over various objectives and 
just about anything beneficial to one side is detrimental to the other. 

Game Theory, and specifically the TPZS game, has been used before to address 
questions related to information — superiority. Dr. Jerome Bracken and 
Dr. Richard Darilek of the RAND Arroyo Center used game theory to examine the value 
of information and its contribution to outcomes.2!_ Their work was expanded by 
Lieutenant Commander Gary McIntosh, United States Navy, as a student at the 
Naval Postgraduate School.?? In both of these research efforts, the research was limited 


strictly to the TPZS game. Additionally, Bracken and Darilek did not incorporate the 


20 Philip D. Straffin, Game Theory and Strategy, The Mathematical Association of America, 1993. 


21 Jerome Bracken and Richard E. Darilek, “Information Superiority and Game Theory: The Value of 
Information in Four Games,” PHALANX, Vol. 31, No. 4, 1998, pp. 6-7, 33-34. 


22 Gary A. McCintosh, “Information Superiority and Game Theory: The Value of Varying Levels of 
Information,” Master’s Thesis, Naval Postgraduate School, Monterey, CA, March 2002. 
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aspect of information quality in their study. McIntosh partly improved on their study in 
this area, but only considered varying information accuracy. While this present research 
seeks to address questions similar to those posed in previous research, the model used 
here—while still drawing on TPZS game principles that will be described in more detail 
in Chapter II—is notably different. 

Applying a game theory construct to these questions represents a significant level 
of abstraction of the situational awareness and decision process described previously. 
Error in situational awareness will be introduced by degrading the information quality, 
while the introduction of error from other sources will be ignored. Additionally, not all 
the contributors to information superiority (ISR; Command, Control, Communications, 
and Computers (C4); IO) will be considered as such, but a general concept of information 
superiority will be applied where the decision maker primarily gains an information 
advantage through the collection of information, while the ability to process and protect 
this information is assumed. The aim is to get at the essence of the contribution of 
information superiority and some of the challenges with achieving information 
superiority, while assuming that the decision maker has the wherewithal to turn this 
information superiority into decision superiority and an actual operational advantage. 

With the background set in place, Chapter II discusses the details of the selected 
modeling approach. Additionally, the mathematical principles involved are laid out with 
a detailed discussion of the model logic. This is followed in Chapter HI by a brief 
explanation of experimental design, how and why it was used in this research, and the 
specifics of the design used for this investigation. Chapter IV covers the analysis of the 
data collected in the experiment, with conclusions and recommendations for future 


studies covered in Chapter V. 


Il. MODEL DESCRIPTION 


A. HOW GAME THEORY APPLIES TO THE MODEL 


The model was developed as an extension of a TPZS game. The scenario is as 
follows: there are three objectives and two sides, red and blue. The three objectives are 
representative of the locations where red and blue will encounter one another in a type of 
engagement. Blue is defending the objectives with its available resources and red is 
trying to maximize the sum of the difference between red and blue at each site. Figure 2 
provides a visualization of this concept (those familiar with TPZS games may recognize 


this as a form similar to a Blotto game.)?3 


Model Starting Point 
Two-Person Zero Sum Game 


C) Max (Red — Blue, 0) @) 




















(nea) @) Max (Red — Blue, 0) @) (Biv) 
Attacker ft Defender 











C) Max (Red — Blue, 0) C) 


— Total Red Score 





Figure 2. Two-Person Zero Sum Game Model Starting Point 


Red, the attacker, will deploy its units to the objectives and blue will do the same. 


Following this action, we can calculate how many more reds are at each location. 


3 To read more about Blotto games see Washburn, A.R., Two-Person Zero-Sum Games, Institute for 
Operations Research and the Management Sciences, December 1994. 
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Summing these values, we get the total red score. Of course, red is trying to maximize 
this score, while blue is trying to minimize this score. Both red and blue will have a 
number of strategies available to them based on their respective number of units. In fact, 


the number of strategies is equal to: 


(# of units + 1) x (# of units + 2) 
2 





Thus, if blue had five units he would have (6 x 7) / 2 = 21 possible strategies to choose 
from. So how do blue and red decide which strategy to select? To answer this question, 
we need to understand more about solving TPZS games. 

A key product to understanding potential outcomes of selecting certain strategies 
is what is known as the payoff matrix. The payoff matrix displays all the outcomes 
associated with the possible strategies for red and blue. Table 1 is a payoff matrix for the 


situation where red has three units and blue has two units, and it shows all the potential 






























































payoffs for red. 
RED/BLUE PAYOFF MATRIX EXAMPLE 
COLUMN A B C D E F 
Blue Strategies 
ROW 002/01 1/02 0/10 1/11 0/200 

A 003] 1 2 3 2 3 3 
B re ee ee | 1 2 3 2 3 
C ~ {0 2 1] 2 1 1 2 2 3 
D | m/03 0; 3 2 1 3 2 3 
E 3 fo 2) 1 2 3 1 2 2 
F|aAtriI1di1 2 1 2 1 1 2 
G/i|3s/120| 3 2 1 2 1 2 
Hi) oo 4a) 2 2 3 1 2 1 
I 210, 3 2 2 2 1 1 
J 30 0| 3 3 3 2 2 1 






































Table 1. | Example Payoff Matrix 


First, observe that red has (4 X 5) / 2 = 10 available strategies, while blue has 
(3 X 4) /2 =6 available strategies. Now let us assume that red selects the strategy in row 
C. This means that red is sending zero units to objective 1, two units to objective 2 and 


one unit to objective 3. Blue, without knowledge as to what strategy red selects, goes 
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with his own strategy of column D, sending one unit to objective 1, zero units to 
objective 2, and one unit to objective 3. If we then examine the intersection of row C and 
column D, we see a value of 2, which is equivalent to the red score calculated as 
discussed previously. Hence, the payoff matrix displays all possible outcomes based on 
the combination of strategies of red and blue. 

In many cases, as in the present situation, there is no single strategy that is clearly 
superior to all the rest. For example, the best possible outcome for red is a score of three, and 
if red had a strategy in which he could score a three every single time, that would be best for 
red. However, more often than not, respective sides must select a mixture of strategies and 
play them with what is called a mixed strategy based on the concept of an expected value, 
where the expected value of getting a certain payoff aj, a2, ...a; with respective probabilities 
Pi, P2, ----Pk iS pray + Prado + ... pra;.24 Within game theory, this reasoning is known as the 
expected value principle, which is “If you know that your opponent is playing a given mixed 


strategy, and will continue to play it regardless of what you do, you should play your strategy 


which has the largest expected value.”25 The reason for this is that there exists a value such 


that red can guarantee it gets, on average, at least this value and blue can guarantee that, on 


average, red gets no more than this value.2® This value is known as the value of the game. 
To solve this TPZS game for the value of the game and determine the optimal mixed 
strategy for red and blue, we apply linear programming techniques. To solve for red, we first 
create a matrix A, by adding a row to the bottom of the payoff matrix with values all equal to 
—1 and then transposing this matrix to get A,. Next, we create a vector x that consists of 
variables X1, X2, ..., X, Where n equals the number of columns in matrix A,. The value of the 


game is x, and the mixed red strategies are the values x), Xo, ...,Xn-1.. Thus, the linear program 


is formed in the following manner2/: 


24 Philip D. Straffin, Game Theory and Strategy, The Mathematical Association of America, 1993, 13. 
25 Tbid., 13. 
26 Tbid., 13. 


27 This particular method was found at a Website operated by Elmer Gerald Wiens, Ph.D. at 
www.egwald.com/operationsresearch/gametheory.php3, accessed January 2008. 
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Maximize: Z = Xy 
Subject to: 
A,x = 0; 


n-l 

y= 0 

i=] 

where all x; are non-negative. 


If it is unclear why this method works, consider for a moment what A;x 
represents. It is a series of equations testing whether the selected optimal mixed strategy 
is greater than or equal to the expected payoff (x,), which we are trying to maximize. 
The optimal mixed strategy must, of course, meet all of these constraints as well as the 
sum of the mixed strategies equaling zero (constraint number two), since each represents 
the proportion of the number of times the specific strategy would be selected. Using the 
example payoff matrix from the previous page, the problem written in long form would 
look like this: 


Maximize: z= X11 

Subject to: 
1x, + 1X> + 2x3 + 3x4 + 1x5 + 2X6 + 3X7 + 2Xg + 3X9 + 3X49 — 1X4] 
2x, + 1X2 + 1x3 + 2x4 + 2x5 4+ 1X6 + 2X7 + 2X8 + 2X9 + 3X19 — 1X11 
3x1 + 2Xo + 1x3 + 1x4 + 3x5 + 2X6 + 1X7 + 3Xg + 2X9 + 3X19 — 1X4] 
2x1 + 3X2 + 2x3 + 3x44 1x5 + 1x6 4+ 2x7 + 1Xg + 2X9 + 2X19 — 1X] 
3x1, + 2X2 + 2x3 + 2x4 + 2X5 + 1X6 + 1X7 + 2Xg 4+ 1X9 + 2X19 — 1x11 
3x1 + 3X2 + 3x3 + 3x4 + 2X5 + 2X6 + 2X7 + 1Xg + 1X9 + 1Xj9— 1x1] 
Xp $X2tX34+X4+X54+X6+X7+Xgt+ Xo + X19 =O 
X1, X2, X3, X4, X5, Xo, X7, Xg,X9, X10 = O; 


. . 


. 


IV IV IV _ IV _IV_IV 
ofofeo29 


To solve this program for the blue mixed strategies we add a column of —1’s to 
the right-hand side of the payoff matrix and then transpose this matrix to create a new 
matrix called Ap. We will use the vector y to represent the combination of blue’s mixed 
strategy and the value of the game. This vector is equal in length to the number of rows 
in matrix Ay. Now, instead of maximizing, we minimize and change the sign on all of the 
constraints and we have the following linear program: 


Minimize: Z= yn 
Subject to: 
Avy < 0; 


n—l 

>. y; = 0; 

i=l 

where all x; are non-negative. 
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Solving this problem will show that y, is equal to X,, which will always be the 


case. More importantly, the y vector will reveal the optimal mixed strategy for blue. 


Using these methods, a payoff matrix, game value, and optimal mixed strategy 
can be computed for any combination of blue and red values. As it turns out, a 
generalization can be made for how both blue and red should behave under the conditions 
described for this scenario. Blue’s strategy is equivalent to trying to defend each 
objective equally. In terms of the output from the linear program, this would mean that if 
blue had seven units, one-third of the time he would play the 3,2,2 strategy, one-third of 
the time the 2,3,2 strategy, and one-third of the time the time the 2,2,3 strategy. On the 
other hand, it is best for red to always to keep the entirety of its units focused on one 
objective. What this means for red, in terms of the mixed strategy, is that red plays 0,0,5 
one-third of the time, 0,5,0 one-third of the time, and 5,0,0 one-third of the time given it 
has five units. These will be the general strategies employed by both red and 


blue, respectively. 


B. EXTENDING THE TWO-PERSON ZERO GAME MODEL INTO OUR 
EXPERIMENTAL MODEL 


The TPZS game model discussed in the previous sections assumes that both sides 
take the optimal action, based on the fact that neither red nor blue can anticipate the 
other’s strategy. However, since we are interested in looking at situations where one side 
obtains an information advantage, we need to create a mechanism that provides 
opportunities for one side to gain information as to its adversary’s intentions. Therefore, 


observe in Figure 3, an illustration we will now refer to as the “game board.” 
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Figure 3. Graphic Depiction of the Game Board 
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On the game board, both red and blue begin with the entirety of their force at their 
respective start locations. There will be a detailed discussion as to the rules of the game, 
but for now understand that red will advance one column per move until it reaches 
column 4. Blue will deploy its forces over time into column five. The final step of the 
game is to calculate the number of red that outnumber the blue in each row between 
columns four and five. The purpose of having all the red movements is to provide an 
opportunity for blue to collect information as to red intentions as red moves using a 
sensor placement strategy. Within this current game, if blue has no ability to collect any 
information on red as red moved to column four, then we have simplified the scenario 
back to the TPZS game discussed in Section A. Hence, the point is to see how blue 
might improve on the outcomes of the basic game if provided a potential information 


advantage under varying conditions. 
C; RULES OF THE GAME 
This section will be a detailed discussion on how the game is played. 


1. Starting Conditions 


It is important to understand where both blue and red start in terms of their 
understanding of the opposition. Remember, we began with the standard TPZS game 
where each side has no knowledge of what the other side will attempt to do. Despite the 
lack of information about their adversary’s specific course of action, but having 
information as to the range of action, each side is able to take some rational action. In 
red’s case, this is to maximize the lowest possible expected value of the outcome; in 
blue’s case, this is to minimize the highest possible expected value of the outcome. Now 
we have modified the situation. Red will not be given any additional information, but 
still understands the basic rules of the game. So, even though we can assume he has 
knowledge as to blue’s ability to collect information on his movement, we will not 
assume he knows to what degree or specificity. Red will stick to the three courses of 
action determined by solving the linear program. From this we conclude that red has no 


decisions to make other than the determination of the optimal mixed strategy to employ 
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over the course of a particular game. We can also state that red begins with knowledge 
about the number of blues and the restrictions of movement that pertain to blue, but 


again, these will not effect the manner in which red plays the game. 


For blue, the situation is quite a bit different. First, blue will also understand the 
restrictions placed on red movement. Additionally, blue will not always have accurate 
knowledge as to the total number of red units. This will vary, based on the accuracy 
variation parameter. Blue will have no knowledge as to what strategy red will employ, 
but will be restricted in trying to determine this, based on the quality of information 
provided by his sensors and the number of sensors themselves. The bottom line is that 
blue will have the opportunity to make adjustments to his strategy. For this reason, blue 


will want to remain as flexible as possible within the rules of the game. 


2. Red Movement Restrictions 


During its first turn, red is required to move the total of its units from the start point 
on the game board to any positions desired in column one. During subsequent turns, it must 
adhere to three important rules governing its movement: 1) during each turn all red units 
must move out of one column and into another, higher-numbered column; 2) across dotted 
paths, red can move a maximum of one unit; and 3) across solid paths, red can move up to 
the maximum number of reds present in the previous location in each of the respective rows. 
This means that after red’s initial deployment, all of its units will be in column four in three 
turns. This is analogous to red re-arraying its forces on the battlefield—a difficult task that 


provides a purpose behind these movement restrictions. 
3. Blue Movement Restrictions 


For blue’s initial deployment from blue start, blue can move as many units into 
column five as desired. We are not interest in testing whether blue can establish its 
defense in time for the arrival of red units, which is the reason blue is allowed to place as 
many units as desired in its initial deployment. Rather, the intent is to examine how 
much can be gained by allowing blue to adjust its defense with the advantage of 


information superiority. However, for each subsequent turn, blue can move only a 
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maximum of three units out of the blue start location. This represents a limitation on the 
amount of flexibility that blue enjoys. Additionally, no blue units can return to the start 
location once deployed. During each turn, blue can move a maximum of one unit 
between each objective in column five. What this means for blue, in practice, is that blue 
will hold back the maximum number of units possible in order to maintain the maximum 
level of flexibility, in the hopes that his sensors will provide some useful information as 
to where red units will be attacking. This is exactly the methodology that was 
implemented in blue’s decision mechanism for deploying forces within the computer 
model. Blue has a priori knowledge of the number of red units but this knowledge is not 


perfect. 
4. Basic Game Play Sequence 


The game is played in the following sequence (implementation of these steps will 


be further explained in Section F). 


1) Blue uses a decision algorithm to place an assigned number of sensors on 
the game board; only one sensor can be placed per location in columns 
one through three on the game board. 


2) Blue uses a decision algorithm to make initial deployment of blue units 
into column five. 


3) Red randomly selects one of its three courses of action to execute not 
knowing blue’s disposition; red moves the total of red units into column 
one in a manner that will still allow it to achieve its selected course of 
action; the assignment of reds to a specific location is a purely stochastic 
process that works within the constraints of the red movement restrictions. 


4) If blue has any sensors located in column one and the information is not 
delayed, this information will be provided to blue and blue will adjust its 
units in column five subject to its movement restrictions. 


5) This process will repeat until the red units are in column four; blue will 
not have any sensors in column four; at this point, a score is calculated for 
red in accordance with the calculation procedure previously discussed, at 
which point the game has ended. 
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D. MODEL PARAMETERS 


The model has five parameters: number of red, number of blue, number of blue 
sensors, accuracy variation, and probability of time delay. The number of red and blue 
should be self-explanatory (however, it should be noted that they are integer values only). 
Red’s will range from 2 to 25 and blue will range from | to (3 * red — 1). Once blue is 
more than three times as large as red, it can defend each location without any concern 
that red will score any points. Thus, it is not important to explore this region. 

Sections D.1. through D.3. describe the reasoning and use of the other parameters 


in detail. 


1. Number of Blue Sensors (Varied from 1 — 9) 


In this experiment, blue is being provided the information advantage in order to 
see how he might improve over the baseline case of information parity. Blue sensors 
have an input range from 1 to 9. Only one sensor can be placed at each of the nine 
discrete locations on the game board, located in columns one through four. Placing a 
sensor at a particular location provides the opportunity for blue to determine something 
about the number of red units at that location at a certain moment in time. Blue is then 
able to use this information to varying degrees to adjust his course of action. However, 
the quality of the information may be impacted by the two parameters, accuracy variation 
and probability of time delay, to be discussed in the following section. The sensor has 
the capability to detect information with perfection. However, the accuracy variation 


parameter provides instances where errors will be introduced to the information collected. 


2. Accuracy Variation (Varied from 1 — 6) 


The quality of information is just as important as the information itself. Poor 
quality information can actually do more harm than good. To incorporate this aspect of 
information quality, both parameters of accuracy variation and probability of delay have 
been introduced. Accuracy variation pertains to the standard error of the normal 
distribution (mean = 1) that is being used to apply a level of error to the number of units 
at a particular location being reported by a sensor. For example, let us say we want the 


error to be added according to a standard normal distribution. To implement this, we 
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would input a one for our accuracy variation parameter. As a result, every time a sensor 
made a detection, a random number would be generated from a normal distribution 
(mean = 0, variance = 1). If a —1.237 is generated, everything to the right of the decimal 
is eliminated (this is not rounding) and a —1 would be added to whichever value the 
sensor detected. Thus, to increase the likelihood of greater error being reported by 
sensors, we increase the accuracy variation parameter. Table 2 provides a sample of how 


the accuracy might vary, based on the accuracy variation parameter. 






































___ | Probability | Probability | Probability | Probability | Probability 
Probability ; : . . | Information 
Accuracy Information | Information | Information | Information|”. 
. .. _ | of Accurate |. : : ; is off by 
Variation . _ | is off by +/— | is off by +/— | is off by +/— | is off by +/- 
Information more than 
1 2 3 4 
+/-4 
1 68% 27% 40% <1% <1% <1% 
Z 38% 30% 18% 9% 3% 2% 
3 26% 23% 19% 13% 9% 10% 
4 20% 18% 16% 14% 11% 21% 
5 16% 15% 14% 12% 11% 32% 
6 13% 13% 12% 12% 10% 40% 
Note: These percentages have been rounded to the nearest percent. 











Table 2. Probability Distribution Based on the Accuracy Variation Parameter 


3. Probability of Time Delay (0.0 — 0.8) 


As you have probably surmised, this parameter—which has a range from 
0 to 0.8—affects the timeliness of the sensor report. A value of 0.5 would mean that 
there is a 50% chance that a delay will be incurred. The length of the delay is always 
only one turn. Thus, a report that blue was expected during his current turn will not be 
available until the following turn. Both the probability of time delay and the accuracy 
variation apply uniformly to each sensor being employed within a specific game (i.e., 
each sensor will have the same probability of delay and the same accuracy variation). 


This was done primarily for the sake of simplicity. 


E. MODEL OUTPUTS 


For each specific set of inputs, the game will run for a predetermined number of 


replications, which will constitute one run of the model. For each run, the model will 
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output a low score, high score, and the average score for red. Figure 4 is a visual 


summary of the inputs and outputs of the model. 





























Inputs Outputs 
# of Red (2 — 25) Red Score 
# of Blue (1 — 74) le NY (0) B) 518 = 
# of Sensors (1 — 9) 
Probability of Time Delay 

(0—0.8 ) 
Accuracy Probability 

(0-6) 

Figure 4. Model Inputs and Outputs 


F. MODEL IMPLEMENTATION 


This model has been implemented in Java with the addition of LPSolve, which is 
a freeware linear program/mixed-integer program (LP/MIP) solver that is compatible 
with Java. The following section discusses specific parts of the implementation of the 
game and provides more details of the specific functionality of the general game 


descriptions provided in Sections C, D, and E. 


1. Red’s Movement 


Red’s movement is governed by a stochastic process and since red is not making 
any decisions based on the progression of the game, all of red’s movements are 
determined at the beginning of the game. Hence, if red begins the game with five units 
and selects the 0,5,0 course of action, we know what the disposition of red in column four 
will be and can build the game board backwards from there. Given that there are a 
positive number of red units at a specific location, there is a fixed probability that red will 
send a red unit across one of the possible dotted lines on the game board. The 
probabilities are weighted toward the location of the preponderance of the red units in 
attempt to get good dispersion of the red units across the game board. Figure 5 contains 


some example game board builds given a 0,5,0 course of action by red. 
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Example C 
Figure 5. Game Board Examples 
The game board is then manipulated to calculate the appropriate score for red 
depending on how blue ends up arraying itself in column five. 


2. Blue’s Sensor Decision Algorithm 


It is important that an effort be made to ensure that blue is using its sensors in the 
best possible manner so as to maximize its informational advantage. In doing so, we 


ensure that poor use of sensors does not confound our results. However, determining an 
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optimal sensor array for a given situation (when there were more nodes than sensors) 
proved an intractable task for this research, even when the information was essentially 
perfect (i.e., no delay and no erroneous information). Using simulation, an attempt was 
made to determine if, for a given number of red, blue, and blue sensors, an optimal sensor 
array could be found. After conducting even 1,000 replications for a given design, the 
vast majority of design points did not converge on an optimal sensor array. Therefore, it 
seemed best to set up a smaller simulation within the overarching model to select an 
appropriate senor array for blue. While the selection may not be optimal, it should, at 
least, be reasonable. This smaller simulation works similarly to the larger simulation, 
however, the difference lies in the fact that for a given number of red, blue, and blue 
sensors, a specific game will be played varying the other two input parameters 
(Probability of Time Delay, Accuracy Variation) stochastically within the smaller 
simulation, while testing every possible sensor configuration available to blue each for 50 
replications. 

By way of example, let us say we are considering a scenario where there are five 
red, five blue, and four sensors. Blue must decide where the four sensors will be placed. 
There are 9 (possible locations among which to choose) choose 4 (locations that will 
actually be selected) possible ways that blue can emplace four sensors. In this case, that 
number equals 126 possible sensor strategies among which to choose. Each of these 126 
possibilities will be run through a smaller simulation of this model 50 times, with an 
average score being computed for each possible sensor strategy. The strategy with the 
highest average will be the one selected by blue to be used against red in the larger game. 
Within each of these smaller simulations, the values of the Accuracy Variation parameter 
and the Probability of Time Delay will be varied stochastically for each and every 
subgame. Two important assumptions will be made concerning the type of distributions 
that apply to each of these varying parameters. First, blue assumes the variation of the 
probability of delay will be between 0 and 0.8 and that accuracy variation will bebetween 
1 and 6 per the parameter ranges of the model. Second, there is a presumption on the part 
of blue that, most of the time, information will be on time and, a majority of the time, the 


information will be close to accurate. 
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Therefore, the distribution of the Probability of Time Delay will follow the 
normal distribution with mean = 0, and standard deviation = 0.15. Any negative values 
drawn from this normal are converted to positive numbers. This means that 65% of the 
time, the probability of delay will be 15% or less, 95% of the time it will be 30% or less, 
and 99% of the time it will be 45% or less. 


The accuracy variation parameter will vary between the values of O and 6 for the 
subgame. The value of the accuracy parameter for a specific sub game will be randomly 
selected based on the algorithm described in Figure 6. 

Draw a uniform random number between 0 and 1; 

Oif U< 0.05 
1if 0.05 <U <0.20 
2if 0.20 < U<0.55 

AccuracyVariation = «3 if 0.55 < U<0.75 
4if 0.75 < U<0.90 
5 if 0.90 < U<0.95 
6if U = 0.95 


Figure 6. Probability Distribution for the Accuracy Variation Parameter Within 
the Subgame 


This represents the idea that 75% of the time blue expects to receive fairly 


accurate information (an accuracy variation parameter less than or equal to 3). 
3. Blue’s Movement Decision Algorithm 


The procedure for determining how blue will move into column five is probably 
the most complex process of the entire model, and involves numerous comparisons and 
updates. As mentioned before, blue desires to maintain maximum flexibility due to the 
fact that blue has the capability to gain information and wants to be in a position to react. 
For this reason, blue always elects to keep the maximum number of blue units at the start 
location as long as possible, provided none are left there at the end of each game. For 
this reason, blue will only deploy the maximum of either the number of blue minus 9, or 


0, during the initial deployment. The 9 comes from the fact that during each turn, blue 
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has the opportunity to move 3 blue from the start location. Since blue has three turns 
after the time of the initial deployment, blue will have the opportunity to deploy the 9 
units not yet committed. The idea is not to commit forces before it is necessary. For 
example, let us say blue begins with 12 units; blue will initially deploy max(12-9,0)=3 
units. Since blue’s general strategy, lacking any other information, is to defend the 


objectives equally, blue will place one unit at each objective in this case. 


For future deployments of blue units, blue will look to update its information 
concerning red’s intentions. To do this, blue keeps track of two pieces of information 
about red: 1) the maximum number of reds possible at each objective, and 2) the 
minimum number of reds required at each objective. These two values are based on red’s 
movement restrictions. Figure 7 provides an analysis of how this calculation works, 


based on the number of reds identified at a certain location. 
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Figure 7. Delta Values Based on Red Locations 


The numbers in columns one through four represent the amount of change that 
can occur (plus or minus) between the number of reds at a current location and the 
number of reds at the final location, respective to each row. Thus, let us say that 15 reds 
are determined to be in row 2, column 2. The delta value for that location is 4; therefore, 
the maximum possible reds that can be in row 2, column 4 is 19. Additionally, there 
must be at least 11 since red can, at maximum, move 4 units from row 2 within the next 
two turns. Now let us consider the case where blue only has one sensor and it is located 
at row 2, column 2 (the location currently under discussion); clearly an update for the 


maximum can be made for the objective in row 2, but what of the others? Well, since 
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blue has calculated the minimum possible at the objective in row 2 to be 11, he can 
subtract this from the total number of reds. If we say the total number of reds is 25, that 
means that there are, at most, 25 — 11 = 14 possible red units at either of the other two 


objectives. 


At this point, we can develop a vector V of the maximum possible red units 
possible at each objective. V; will represent the maximum red units possible at the row 1 
objective and so on. It is this vector that governs blue deployments as a weighting 
scheme. If V, through V3 are equal (as is the case at the time of initial deployment when 
they all equal the total number of reds), then each objective has equal weight and blue 
attempts to defend each equally. If this vector remains unchanged throughout each turn, 
blue will evaluate the delta between the maximum possible at a particular objective and 
the current number of blues deployed to that objective. Blue seeks to keep these deltas at 
equal values. So, if blue has more units deployed to a certain location above the 
maximum number possible, it will begin to move those units to other locations—the 
priority being to locations with the largest delta between the maximum number of reds 
possible and the number of blues deployed to that location. It is probably helpful to go 
through an example. Let us consider a situation where red has 7 units, blue has 12 units, 
and blue has 2 sensors. This situation is depicted in Figure 8, where the squares represent 


the locations of the blue sensor. 
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Game Board Depicting Location of Blue Sensors, Number of Red Units, 
and Number of Blue Units 


In this example, blue makes its initial deployment and then red does the same 


with the result seen in Figure 9. 
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Figure 9. 
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Depiction of Game Board Following Initial Deployments 


If blue had any sensors in column one, he might be able to make an update to 


vector V; as it is, the vector remains (7, 7, 7) and blue makes his next deployment shown 


in Figure 10. 
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Depiction of Game Board Following Turn One 


Now, assuming there was no delay and the information was accurate, a blue 
sensor picks up the fact that there are five red units located in row three and column two. 
Due to the movement restriction placed on red, there are a maximum of seven red units 
that can show up at the objective located in row three. Therefore, blue will not change 
the value of V3. However, V,; and V> can both be decreased since the minimum value of 
red units possible at the objective located in row three is three. That is, three red units 
that cannot be used at the other two objective locations. Thus, blue can reduce V; and V2 
by three. This means V now equals (4,4,7). As a result, of the next three units that blue 
deploys, two will go to row three, column five and the last one will be randomly 
distributed among the three. Thus, as blue makes its next deployment and red makes its 


next move, the game board could look like Figure 11. 
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Figure 11. Depiction of Game Board Following Turn Two 


In this next step, blue again has a sensor in play and determines that there are zero 
red units located at row one, column three. The update, in this case, is only made to V; 
since the minimum value is of no value (it equals zero in this instance) and V becomes 
(1, 4, 7). Blue will now try to move blue units away from the objective in row one 
because it determines it has too many, and then the final three units will be deployed. It 
can only move one away per turn, so blue will finish this game with two blue units at the 
objective in row one. Again, the first two blue units deployed will go to row three in 
order to even up the gap between V and the current disposition of blue units. The final 
blue unit will be randomly assigned between rows two and three since these are the only 
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locations blue deems in need of additional units. Thus, the game board could turn out as 
displayed in Figure 14. In this case, it turns out red will get a score of 1, although it was 
possible that red could have scored zero if the final blue unit would have gone to row 


three. 
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Figure 12. Depiction of Game Board Following Turn Three 


Of course, all of these calculations are being influenced by the Accuracy 
Variation and the Probability of Time Delay parameters. If there is a delay in blue 
gaining information, blue loses the opportunity to take advantage of the potential value of 
that information. However, prior to the next turn, vector V is updated first, based on the 
delayed information, and then it is updated based on the availability of any new 
information, and then blue makes its next deployment of units. There is no direct 
resolution for any potential conflicting or ambiguous information. By virtue of how the 
algorithm is constructed, some deference is given to the latest information, but there is no 
direct attempt to resolve any potential ambiguity. For example, given a delay from 
column 2 to column 3 for a specific location, blue will first adjust his disposition based 
on the delayed information and then use the current information. However, let us say that 
the delayed information indicated the presence of 10 red in the previous location and the 
current information says that there are now 2 red in the subsequent location. Blue would 


conclude that one or both of the sensor information provided has an error based on the 
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red movement restriction. Red could never decrease by 8 units between two locations in 
the same row. However, this model does not attempt to resolve this observable error but 


simply allows the algorithm to continue. 


With the ground laid in understanding the model, Chapter HI will discuss the 
experimental design meant to explore the model and move the study toward the 


development of interesting insights concerning information superiority. 
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HI. EXPERIMENTAL DESIGN 


A. INTRODUCTION 


With our model fully implemented using the JAVA computing language, we now 
want to take advantage of modern computing power to run the simulation many times 
across the spectrum of input parameters. The purpose of experimental designs is to help 
us explore our model more efficiently.28 When dealing with stochastic models with even 
a small number of input parameters, exploring the “Landscape of Possibilities”—to steal 
the theme from the 2008 Data Farming Workshop—can be a challenging task. Where are 
the points of interest and what areas require a more detailed search? How is a simple 
overview of the “landscape” accomplished? 

Let us review the input parameters and ranges for each parameter of our model, 


which was discussed in detail in Chapter II. Table 3 lists the five input parameters and 
































their ranges. 
Parameter Range __|Decimals 
Red Units Da2d 0 
Blue Units 1 - 74 0 
Blue Sensors 1-9 0 
Probability of Time Delay | 0.00 - 0.80 Z 
Accuracy Variation 0.0 - 6.0 1 





Table 3. Table 3. List of Model Parameters with Respective Ranges 


An important point to make about the number of blue units is that for a given 
number of red, the blue units will only range from 1 to (3 * number of red — 1). Thus, 
there are 950 possible combinations of red and blue units. From Figure 15, we see that 
there are nine possible variations for the sensors, 81 for the probability of time delay, and 
60 for the accuracy variations. This means that there are 950 * 9 * 81 * 60 = 41,553,000 


different design points or distinct inputs variations that we can input to the model. If we 


28 Susan M. Sanchez, “Work Smarter, Not Harder: Guidelines for Designing Simulation 
Experiments,” Proceedings of the 2006 Winter Simulation Conference, pp. 47-57. 
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wanted to do only ten replications on each design point, we would exceed 400 million 
runs of our model. That is not only a lot of data to analyze, but it may exceed available 
computing resources.2? Therefore, we apply a design of experiments. 

A specific design is a matrix of the design points that will be input into the model 
where the columns of the matrix are variations of specific parameters and each row 
contains the combination of parameter values for a specific design point. Ideally, the 
design points would be spread apart from each other and the columns of our matrix 
would be orthogonal, allowing us a better opportunity to explore the multidimensional 
space. By way of a simple example, let us say we are trying to map a certain section of 
the ocean floor; a three-dimensional space. A single data point will give us an x, y, and z 
coordinate for a specific location. If we only had the ability to collect ten data points, we 
would want an optimal spread of those ten points to get the best overview of the terrain 
possible. As we increase the number of data points collected, it should be obvious that 
the resolution of our map of the ocean floor increases. However, we want to increase the 
resolution uniformly across the entire space; thus, we want to maintain this optimal 
spread of our data points. This is the concept behind the use of nearly orthogonal Latin 
hyper-cubes (NOLH) to build an experimental design that has good space-filling 
properties, while keeping the correlation between the columns to a minimum.?? 
Additionally, it helps us avoid having large unpredictable slopes or discontinuities. A 
review of the references will provide a more detailed explanation of both the conceptual 
underpinnings of experimental design and the mathematics associated with building an 


NOLH. 


B. DESIGN SPECIFICS 


This experiment was built on a crossed-design method, implying a crossing of 
two design methodologies. Due to the peculiar relationship between the number of blue 


to the number of red, a complete enumeration was used for the combination of these two 





29 In the case of this experiment, the design being applied took less than five hours which represented 
1,564,200 runs of the model. However, the simulation was run on a cluster of 32 computers, which greatly 
reduced the time required. 


30 Thomas M. Cioppa and Thomas W. Lucas, “Efficient Nearly Orthogonal and Space-Filling Latin 
Hypercubes,” Technometrics, Volume 49, Number 1, February 2007. 
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parameters. This means that the rest of the experiment was built around the 
948 combinations of red and blue. For each of these combinations, an NOLH design was 


applied using the remainder of the parameters. The NOLH design is shown in Table 4. 











low 0 
high 0.8 
decimals 0 2 
factor AccVar AccVar AccVar 





1.5 : 5.3 : 2.8 

6 ; 2.8 : 1.9 
3.2 : 0.2 : 0.2 
5.4 : 5.6 ‘ 0.4 
1.2 : 4.1 ; 3.4 
5.7 ; 2.6 : D3 
3.3 ; 0 ; 5.4 
44 : 5.4 
2.3 : 4.3 
4.3 ; 1.3 
2.1 : 2.1 
4.6 é 4.5 
1.8 : 3.6 
0.9 











































































































Table 4. NOLH for 3 Model Parameters 
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There are 99 design points that have been placed in adjacent columns of 33 each 
for the convenience of showing the design all on one page. Each of the 99 design points 
for Number of Sensors, Accuracy Variation, and Probability of Time Delay, were 
combined with each combination of red and blue to create 948 x 99 = 93,852 design 
points. Each of these design points was replicated 50 times for a total of 4,692,600 data 
points at the end of the experiment. A full design would have meant 35,245,692 design 
points alone. 

Before moving on to the analysis of the data, we can examine a few items 
regarding our experimental design. First, let us observe the correlation of the 
three-parameter NOLH from Table 4. Table 5 shows the output from Excel '97 when 


calculating the correlation between each column of the respective factors. 


Column1 Column2 Column 3 


Column 1 1 


Column 2 —0.00113 1 
Column 3 0.01094 —0.00279 





Table 5. Column Correlation of NOLH 


Observe that the correlation between the three columns is nonzero, but it is very 
low, which is what we desire. Second, Figure 13 shows a scatterplot matrix of the input 
parameters for the 31,350 design points. We want to focus on the last three blocks that 
display how the 33 points of the NOLH fill out the design points. What we observe is 
that the 33 points have a good spread across the design space. Also note the gapped 
areas, which is not necessarily a point of concern. This gapped space could be filled in 
with additional design points by rotating the design if desired, but is not essential— 
depending on the conclusion trying to be drawn. Moving forward with our experimental 


design, Chapter IV will discuss the analysis of the data farmed in this experiment. 
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Figure 13. Scatterplot Matrix of Input Parameters for Experimental Design 
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IV. DATA ANALYSIS 


A. SETTING THE COMPARATIVE BASELINE 


Recall from Chapter II that central to our model are the concepts of a two-person 
zero sum (TPZS) game. Also discussed in Chapter II is the fact that TPZS games can be 
solved mathematically to determine a game’s value, which is equivalent to the expected 
outcome of the game if each player chooses their respective optimal strategy. This 
average, or expected, outcome will serve as the basis of comparison as we prepare to 
examine the effects on the game when blue is allowed an increasing number of sensors. 
Keep in mind, the effects of information quality represented by the probability of delay 


and accuracy variation parameters will also be taken into account. 


In the context of our model, the TPZS game serves as the example of the situation 
where information parity exists between to the two players (red and blue). Figure 14 is a 
compilation of all the game values given a specific number of red and a specific number 
of blue. The number of red units is explicitly labeled on the x-axis, while the set of data 
points over a specific red unit label refer to the game values associated with the 
respective number of red units and an increasing number of blue units within each 
grouping. This graphical technique will be applied regularly in the explanation of the 
results. Figure 14 explains how to infer the range of blue units associated with a given 


number of red. 


The analysis of the data will include the use of linear regression models and 
graphical representations of the data provide by the JMP and Microsoft Excel °97 


software, with conclusions to follow in Chapter V. 
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B. LINEAR REGRESSION MODEL 


Linear regression models are an excellent tool for assessing the impact of each 
parameter, examining the amount of variation within a simulation, and for gathering 
insights about the interactions among the various model parameters. In the case of this 
experiment, we have five model parameters: the number of red units, the number of blue 
units, the number of sensors (for blue), accuracy variation, and probability of time delay. 
The predictor variable used is a value labeled the improved percent decrease in red units. 


This value is calculated as follows: 


Improved % Decrease = Expected % Reduction to Red Force - % Reduction to Red Force 


where 


Number of Red - GameValue 
Number of Red 





Expected % Reduction to Red Force = 


and 


Number of Red - Red Score 
Number of Red 


% Reduction to Red Force = 





An equivalent mathematical expression is: 


Red Score - Game Value 


Improved % Decrease = 
Number of Red 


This measurement will be used numerous times throughout. Applying a percentage, 
rather than examining raw red score values, seemed to more meaningful when looking 
over a variety of scenarios where the size of red and blue were varied significantly. It is 
also important to note that the regressions were not accomplished using the more than 
4.5 million data points representing the replication of each of the 93,582 design points 50 
times, but an average red score was computed for each design. Since we are focused on 


identifying significant trends in the data, this also seemed a reasonable maneuver. 
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A variety of interaction and polynomial levels were examined using the step-wise 
regression feature within JMP. Regression models that incorporated third order 
polynomials and three-way interaction yielded the best results. Figure 15 is a summary 
of the regression model that was fit to the data. With 29 model parameters and an 
adjusted R-square almost in line with the actual R-square, this is a very good model fit. 
The model accounts for 93% of the variation in the Improved % Decrease, and all terms 


are highly significant (p-value<.0005). 
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Actual by Predicted Plot 


























0.5 
i as Summary of Fit 
&— 0.2 
: z m RSquare 0.930949 
5 a4 a. RSquare Adj 0.930928 
ad yon Root Mean Square Error 0.028289 
a0 G00 "a2 "0% | Mean of Response 0.210334 
P<.0001 RSq=0.93 RMSE-0.0283 Observations (or Sum Wgts) 93852 
Parameter Estimates 
Term Estimate Std Error t Ratio Probs>|t| 
Intercept 1.2081456 0.006534 184.91 0.0000* 
NumRed -0.042858 0.000373 -114.9 0.0000* 
NumBlue 0.0273653 0.000244 112.17 0.0000* 
BlueToRed -0.503757 0.0042 -120.0 0.0000* 
NumSensors 0.0046076 0.000106 43.59 0.0000* 
ProbDelay -0.095354 0.000569 -167.5 0.0000* 
AccVar -0.018676 8.054e-5 -231.9 0.0000* 
(NumRed- 17.1392) *(NumBlue-25.7089) -0.001619 1.571e-5 -103.1 0.0000* 
(NumRed-17.1392)*(NumSensors-5.12121) -0.000705 1.432e-5 -49.26 0.0000* 
(NumRed- 17.1392) *(ProbDelay-0.40242) 0.004666 0.000145 32.19 <.0001* 
(NumRed-17.1392)*(AccVar-3.17172) 0.0022858 2.023e-5 112.97 0.0000* 
(NumBlue-25.7089) *(BlueToRed- 1.5) -0.019267 0.000168 -114.7 0.0000* 
(NumBlue-25.7089) *(NumSensors-5.12121) 0.0005509 + 9.257e-6 59.51 0.0000* 
(NumBlue-25.7089) *(ProbDelay-0.40242) -0.004381 9.367e-5 -46.77 0.0000* 
(NumBlue-25.7089)*(AccVar-3.17172) -0.00063 0.000013 -48.17 0.0000* 
(BlueToRed-1.5)*(NumSensors-5.12121) -0.012117 0.000166 -72.93 0.0000* 
(BlueToRed- 1.5) *(ProbDelay-0.40242) 0.097274 0.001681 57.86 0.0000* 
(BlueToRed-1.5)*(AccVar-3.17172) 0.010758 0.000235 45.83 0.0000* 
(NumBlue-25.7089) *(BlueToRed-1.5)*(NumSensors-5.12121) -0.000345 3.268e-6 -105.5 0.0000* 
(NumBlue-25.7089)*(BlueToRed-1.5)*(ProbDelay-0.40242) 0.003009 0.000033 90.98 0.0000* 
(NumBlue-25.7089)*(BlueToRed-1.5)*(AccVar-3.17172) 0.0003579 4.617e-6 77.51 0.0000* 
(NumRed-17.1392)*(NumRed-17.1392) 0.0002403 9.642e-6 24.92 <.0001* 
(NumRed- 17.1392) *(NumRed- 17.1392)*(NumRed- 17.1392) 0.0000604 4.807e-7 125.66 0.0000* 
(NumBlue-25.7089) *(NumBlue-25.7089) 0.0006509 6.096e-6 106.78 0.0000* 
(NumBlue-25.7089)*(NumBlue-25.7089)*(NumBlue-25.7089) -1.905e-6 3.382e-8 -56.31 0.0000* 
(BlueToRed- 1.5)*(BlueToRed- 1.5) -0.003943 0.001107 -3.56 0.0004* 
(BlueToRed- 1.5) *(BlueToRed- 1.5)*(BlueToRed- 1.5) 0.0171338 0.00031 55.20 0.0000* 
(NumSensors-5.12121)*(NumSensors-5.12121) -0.002228 1.735e-5 -128.4 0.0000* 
(NumSensors-5.12121)*(NumSensors-5.12121)*(NumSensors-5.12121) 0.0008042 8.388e-6 95.87 0.0000* 


Figure 15. Regression of % Decrease in Red Units vs. Model Parameters 


We observe that all of the parameters are included in the first order effects and 
they are having the impact we expect on the improved percent decrease (IPD); increasing 
red units decreases IPD, probability of delay and accuracy variation decreases IPD, 


increasing blue units and number of sensors increases the IPD. This model also 
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incorporates a blue-to-red ratio that is computed by dividing the number of blue units by 
the number of red units. Intuition simply led to the inclusion of this parameter and it 
turns out that it is significant. Nothing stood out in terms of the interaction terms or 
polynomial terms, but the next question is, of course, how substantial an impact do these 
terms have on the IPD? To answer this question, we turn to the estimates scaling feature 
within JMP. Figure 16 displays the scaled coefficient estimates of the linear 


regression model. 


Scaled Estimates 
Continuous factors centered by mean, scaled by range/2 

















Scaled 
Term Estimate Std Error t Ratio Probs|t| 
Intercept 0.3474839 0.000226 1534.36 0.0000* 
NumRed -0.492861 0.004289 -114.92 0.0000* 
NumBlue 0.9988335 0.008905 112.17 0.0000* 
BlueToRed -0.735485 0.006131 -119.96 0.0000* 
NumSensors 0.0184305 0.000423 43.59 0.0000* 
ProbDelay -0.038141 0.000228 -167.49 0.0000* 
AccVar -0.056029 0.000242 -231.89 0.0000* 
(NumRed-17.1392)*(NumBlue-25.7089) -0.679629 0.006594 -103.07 0.0000* 
(NumRed-17.1392)*(NumSensors-5.12121) -0.032452 0.000659 -49.26 0.0000* 
(NumRed-17.1392)*(ProbDelay-0.40242) 0.0214634 0.000667 32.19  <.0001* 
(NumRed-17.1392)*(AccVar-3.17172) 0.0788608 0.000698 112.97 0.0000* 
(NumBlue-25.7089) *(BlueToRed- 1.5) -1.026712 0.00895 -114.72 0.0000* 
(NumBlue-25.7089)*(NumSensors-5.12121) 0.0804325 0.001351 59.51 0.0000* 
(NumBlue-25.7089) *(ProbDelay-0.40242) -0.063958 0.001368 -46.77 0.0000* 
(NumBlue-25.7089) *(AccVar-3.17172) -0.068975 0.001432 -48.17 0.0000* 
(BlueToRed-1.5)*(NumSensors-5.12121) -0.070761 0.00097 -72.93 0.0000* 
(BlueToRed-1.5)*(ProbDelay-0.40242) 0.056808 0.000982 57.86 0.0000* 
(BlueToRed-1.5)*(AccVar-3.17172) 0.0471198 0.001028 45.83 0.0000* 
(NumBlue-25.7089) *(BlueToRed-1.5)*(NumSensors-5.12121) -0.073513 0.000697 -105.52 0.0000* 
(NumBlue-25.7089)*(BlueToRed-1.5)*(ProbDelay-0.40242) 0.0641396 0.000705 90.98 0.0000* 
(NumBlue-25.7089) *(BlueToRed-1.5)*(AccVar-3.17172) 0.0572128 0.000738 77.51 0.0000* 
(NumRed-17.1392)*(NumRed-17.1392) 0.0317803 0.001275 24.92 <.0001* 
(NumRed-17.1392)*(NumRed-17.1392)*(NumRed- 17.1392) 0.0918711 0.000731 125.66 0.0000* 
(NumBlue-25.7089) *(NumBlue-25.7089) 0.8671578 0.008121 106.78 0.0000* 
(NumBlue-25.7089) *(NumBlue-25.7089) *(NumBlue-25.7089) -0.092617 0.001645 -56.31 0.0000* 
(BlueToRed-1.5)*(BlueToRed-1.5) -0.008405 0.00236 -3.56 0.0004* 
(BlueToRed-1.5)*(BlueToRed-1.5)*(BlueToRed-1.5) 0.0533228 0.000966 55.20 0.0000" 
(NumSensors-5.12121)*(NumSensors-5.12121) -0.035649 0.000278 -128.40 0.0000* 
(NumSensors-5.12121)*(NumSensors-5.12121)*(NumSensors-5.12121) 0.051468 0.000537 95.87 0.0000* 
Figure 16. Scaled Estimates of the Linear Regression Coefficients 


This graphic displays in which direction (positive or negative), and to what 
degree, a particular parameter coefficient is having on the response, which, in this case, is 
the IPD. This feature was actually used to eliminate about 12 other parameters that were 
originally included by the JMP software. However, removing these 12 parameters 


dropped the R-square by only 0.00261. Examining Figure 16, one might argue that even 
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more parameters could be removed. Nonetheless, the figure’s message clearly comes 
through that force sizes and force ratios are far and away the most important factors. In 
fact, taking the top six scaled estimates (number of red, number of blue, blue-to-red ratio, 
blue * blue-to-red ratio, number of red’, and number of blue’) to build a regression would 
yield an R-square of 0.71. Noting the diminished impact of the sensors on the outcome, 
we still want to measure the degree of its impact, broken down by sensor, if possible. To 


do this, we will turn to some visual aids to assist in this endeavor. 
C. IDENTIFYING GRAPHICAL TRENDS 


With only five model parameters, we have more flexibility to use traditional 
graphs and plots to aid in visualizing the data. Figure 17 is a very simple summary of the 
results showing the red score for a given number of red and blue averaged across all the 
variations of the other three parameters. These new data have been overlaid on top of the 
data from Figure 14 to show the improvement of the more complicated game over the 
expected score for a simple TPZS game. Recall that a TPZS game is without sensors, 
and the more complicated game involves multiple sensors and takes into account 
information quality. Observe that for all instances where the number of red units is 
greater than five, blue dominates red with the use of sensors in terms of improving over 
the previous expected value of a TPZS game. Even in cases where red has less than five 
units, the blue side still does well in many instances, although note that the margin of 
improvement is noticeably diminished. Also note that the improvement is not uniform, in 
which case the lines would be parallel. In fact, we observe that when blue is at its least 
amount, the improvements are slim; they steeply increase until a point where the 
increased improvement with the increase in blue units becomes more modest. Finally, 
another turning point is reached where the degree of improvement begins decreasing 
again until blue is at its maximum number; the increase from sensors is practically 
nothing. This suggests the varying importance of information while supporting the 
observation in the previous section concerning the importance of force size and force 


ratio. 
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Figure 17. 


Comparison of Red Scores with Sensors to Red Scores without Sensors 
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Figure 18 characterizes the percentage difference in red scores shown in Figure 
17. Figure 18 shows much more clearly the steep increase in the improved percent 
decrease initially enjoyed by blue and then the plateau at the top, where the 
improvements are more modest. Finally, the drop-off is as dramatic as the improvement. 
Also noted on the figure are the estimated values of blue associated with these plateaus. 
Once the number of red units increases to 13, the magic number for blue seems to be 10, 
at which point, blue enjoys a significant degradation to the red score until the number of 
blue units reaches about one and half times that of the number of red units. This again 
reinforces the force size observation, but it also begins to quantify the degree to which 
red is being degraded. Observe that for the peak cases, blue is reducing red anywhere 
from 22% to 33%. However, for most of the cases, blue is reducing the red score by less 
than 20%, particularly in the situation where red has less than 13 units. This also partly 
explains why the number of sensors was shown to have a minimal impact in the 


regression model. 


43 















































0.4 
s 
z 03 
3 £02 
ee 
uo} 
0.1 
i 2 
EF 0 
-0.1 : 
Number of Red Units 
a 
2 
5 
5 2 
as 
ie 
uo} 
E [a4 
a. 
5 
Number of Red Units 
= 
oO 
3 
3 2 
ee 
2 
Ke) 
S a4 
a 
& 
21 22 23 24 a 
Number of Red Units 
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D. VALUE OF VARIOUS LEVELS OF SENSORS 


The sensor parameter was varied from one sensor up to nine sensors. Blue having 
nine sensors means that blue had a sensor located at every possible red location. This 


section will examine the various performances of the nine different levels independently. 


Figure 19 displays contour plots of the data for each sensor, while varying the 
blue-to-red ratio and the number of blue units, which we already know are very important 
to the improved percent decrease to red. The contour plots are useful for pointing out 
critical areas. They are similar to viewing a topographic map, where lines that are close 
together represent a rapid change in terrain and lines further apart represent more gradual 
change or plateaus. We are looking for clear patterns related to in the levels for each 
sensor, as well as differences among the sensors. Note that the contour plots of Figure 19 


(on the following page) are scaled uniformly for ease of comparison. 


The first thing to note is the similarity between the locations of the plateau on all 
the plots. All of the plateaus rest roughly between a blue-to-red ratio of two-thirds to two 
and between 10 blue units and 40 blue units. This coincides with our observation in 
Section C concerning the range where blue gets maximum value from its sensors. The 
second thing to note is the difference in the plots. They all have roughly the same shape, 
but a significant color change occurs between one and two sensors, indicating a 
significant jump in performance between these two levels. We observe a slight change 
between two and three sensors, three and four, and four and five. However, once we 
reach five sensors, there is hardly any noticeable change between any additional sensors 
after five. This suggests that we have reached the maximum potential of our information 
advantage at five sensors. Thus, five sensors appear to do as well as the scenario where 
there is a sensor at every location. We will break this out further to capture the value of 


these sensors in terms of the number of blue units. 
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Contour Plots by Sensor for Number of Blue Units vs. Blue to Red Ratio 
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Given the information above, we can get another perspective with a little more 
detail on the impact of varying sensor levels using the line graph methodology employed 
previously. Figures 20 and 21 are very similar to Figure 18 in terms of shape, except for 
two things: 1) each level of sensor is shown independently and 2) the y-axes are different 
in the case of Figure 21. Figure 20 is somewhat of an addition to Figure 18, showing the 
breakout of the value of each sensor in terms of the improved percent reduction to red 
score. For Figure 21, the blue unit value on the y-axis was calculated by determining the 
number of blue units required to achieve a similar expected value to that achieved by the 


use of the respective number of sensors. 


The first thing to note is that the size of the jump in performance between the 
various sensor levels is readily apparent in both figures. As with the contour plots, we 
see a sizeable jump in performance even with only one sensor. Additionally, we can 
generally observe that the results are fairly consistent for two, three, or four sensors—at 
least until the number of red units is equal to 20. Also, while there may be a statistical 
difference between five or more sensors, there is no practical importance when a 
conservative estimate for that difference is something less than a 2% improved decrease. 
All of these observations conform to those of the contour plots as shown previously, 
although a little bit more variation is visible with the detailed breakout. More 
importantly, however, is the relative comparison that can be made between the number of 
sensors and the number of blue units. Figure 20 should be read by considering a situation 
where there are 12 red units and 10 blue units. In this scenario, if we want to determine 
the relative values of sensors, we find the set of data respective to 12 red units, identify 
the approximate location of 10 blue units, and read the values from the y-axis for each 
sensor value. In this case, 1 sensor is equal to about six blue units; 2, 3, and 4 sensors 
between 7 and 9 blue units; and 5 or more sensors about 12 blue units. Within this 
abstraction, it is difficult to determine whether one sensor is worth six blue units, but 
these are the types of comparisons we want to be able to draw when considering real 
equipment. However, we once again observe that costs are not uniform across the board. 
The conditions of this model suggest that a good strategy involves building force size 
initially, adding some sensors, and then adding more force size which equates to a 
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nonlinear function of blue units and blue sensors mixes. Table 6 shows various side-by- 
side slices taken out of Figures 20 and 21 when the number of red is equal to 17 for 
various numbers of blue to show these ideas more clearly. The equivalent blue is equal to 
the increase in the number of blue required to achieve the same percent improved 
decrease that is listed above it. Thus, in the case of where blue has 10 units and 1 sensor 
the percent decrease is 14%. If blue wanted to have a similar effect purely by increasing 
the number of blue he would need an additional 7.2 blue added to his current number of 
17. Moving across a row, notice how the percent improved increase goes up until the last 
column, where in all cases the value goes down. Also notice that as we move down a 
column there is a noticeable jump in value from | to 2 sensors but everything tops out at 


5 sensors at which point no large increases are made. 











Number of Red = 17 Number of Blue 
Sensors Value Type 6 10 17 27 40 
1 % Imp Decrease 9.8% 13.3% 14.0% 19.2% 15.7% 


% Imp Decrease 18.5% 23.2% 25.8% 31.8% 19.0% 


N 


3 % Imp Decrease 19.0% 26.2% 266% 30.5% 18.9% 


4 % Imp Decrease 20.0% 27.5% 29.5% 32.6% 19.5% 


% Imp Decrease 22.0% 28.9% 32.2% 36.7% 20.5% 


m1) 


6 % Imp Decrease 21.5% 27.7% 31.0% 37.0% 20.7% 


% Imp Decrease 21.7% 27.8% 31.2% 365% 204% 


~“I 


8 % Imp Decrease 21.7% 29.0% 33.1% 37.8% 20.6% 





% Imp Decrease 21.7% 28.7% 32.6% 36.7% 205% 





‘oe 


Table 6. | Comparing Sensor Performance with Equivalent Number of Blue Units 
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Figure 20. 


Improved Percent Decrease in Red Score by Sensor per Number of Red and Number of Blue 
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Figure 21. 


Sensor Value in Terms of Blue Units by Sensor per Number of Red and Number of Blue 
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E. ASSESSING RISK AND THE IMPACT OF INFORMATION QUALITY 


One of our immediate intuitions concerning the heavy use of information is that 
there will be times when performance will drop below a certain threshold that would not 
have been possible otherwise, since we leave ourselves open to deception or simple 
human errors on the friendly side. While the results show a general improvement to the 
expected value, this only represents the expected average—it does not take into account a 
worst-case scenario. Within the context of the TPZS game, if blue plays his optimal 
strategy, he can guarantee a maximum red score. This maximum red score can be 
calculated by determining the minimal number of blue units possible at any one of the 
three objectives. In trying to establish information superiority, it is possible for blue to 
deviate from his original strategy, thereby giving opportunity for the red side to surpass 
this minimum number. This is the inherent risk in pursuing information superiority. For 
this experiment, we can calculate the number of instances when this occurred. 

Conversely, in light of this risk, there is also the potential for increased gain. It is 
in realizing this potential gain that blue is able to improve upon the original expected 
value. Again, going back to the TPZS game, in addition to a minimum red score, there is 
also a maximum decrease that is possible when blue is using his optimal strategy. Unless 
blue is able to achieve some other advantage, blue will never be able to improve upon 
this value. Yet, this is what an information advantage is able to do. It allows blue to 
capitalize on degrading the red capability. 

Figure 22 captures this information succinctly by segmenting the data by the 
number of sensors, and then showing how many data sample scores were below the 
minimum and how many beat the maximum. Overall, we observe that about 2% of the 
sample proportion was below the minimum, while about 85% of the proportion was 


above. That alone is evidence that while there is risk, the gains appear worth the risk. 
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Frequencies Frequencies Frequencies 


Level Count Prob Level Count Prob Level Count Prob 
1 284400 0.06061 1 5524 0.05396 1 218304 0.05451 
2 568800 0.12121 2 17355 0.16954 2 475973 0.11884 
3 568800 0.12121 3 21985 0.21477 3 470587 0.11750 
4 568800 0.12121 4 19543 0.19092 4 474533 0.11849 
5 568800 0.12121 5 7860 0.07679 5 497323 0.12418 
6 568800 0.12121 6 7831 0.07650 6 498258 0.12441 
7 568800 0.12121 7 7620 0.07444 7 498251 0.12441 
8 568800 0.12121 8 8128 0.07940 8 499180 0.12464 
9 426600 0.09091 9 6517 0.06367 9 372591 0.09303 
ee 4692600 1.00000 Total 102363 1.00000 Total 4005000 1.00000 
, ee : N Missing 0 N a 0 
9 Levels evels 
Total Number by Sensor — Total Number Below Total Number Above 
Figure 22. Total Samples by Sensor, Samples Below Minimum, Samples Above 
Maximum 


Table 7 summarizes the data from Figure 22 into a comparative. From it we see 


there appears to be no particular advantage in terms of the number of sensors in 


considering risk or gain. 


% Below % Above 
NumSensors Min Max 
1 ; 


Table 7. | Comparison of Risk vs. Gain by Sensor 
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In considering the effect of information quality on the results, we again turn to 
contour plots. Figure 22 plots the probability of time delay by the accuracy variation 
parameter for each particular sensor. First, note that our intuition is confirmed— 
decreasing information quality results in a decreased effect of our sensors. In this case, 
as we increase the probability of time delay and the accuracy variation, the quality of our 
results decrease. Interestingly, the slope remains fairly steady and constant for all levels 
of sensors. There appear to be no points where once the information gets so bad, there is 
a significant drop in the improved percent decrease. The plots also suggest that 
increasing the number of sensors can overcome some of the effects of degraded 
information. However, recall the observations from the scaled estimates of the linear 
regression model indicate that, while probability of time delay and accuracy variation are 
included in the model, their impact is limited. It is possible that these two parameters are 
simply being overrun by the other information in the model. Thus, it appears we cannot 
draw any strong conclusion from the contour plots. Chapter V will go on to summarize 


the major findings and discuss potential future research areas. 
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Figure 23. 
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Contour Plots by Sensor of Probability of Delay vs. Accuracy Variation 


Vv. CONCLUSIONS 


A. SUMMARY OF FINDINGS 


For the given circumstances of this model, the data have shown that the value of 
information superiority is not uniform, but is strongly influenced by force ratio and force 
size. This does not suggest that information superiority has no value; on the contrary, we 
measured the added value in this study. However, it does suggest the possibility that our 
zeal for information superiority should be tempered by the fact that force size plays a 
significant role as well. There are “knees in the curve” at both ends of the force size 
spectrum. First, force size must be increased to a certain level before information 
superiority can be utilized to its maximum extent. Second, once a certain force level is 
reached, information superiority begins to decrease in value. This is another reminder of 
the importance of striking the balance between sensor and shooter and why this area of 
research is so important. Additionally, this relationship suggests that information 
capabilities should not be viewed as a simple add-on to force capability, but that the 
values of force size and information are dependent on one another. Thus, force 
development must incorporate and evaluate the combined capabilities of information 
systems and combat equipment, and not assess these capabilities individually. 

This study also suggests that information superiority can be increased to a point 
after which additional information superiority translates into little or no value in decision 
superiority on the battlefield. A chart of the general value per sensor appears in Figure 


pies 
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Figure 24. General Value of Information Superiority by Sensor 


Specifically, we see that increasing the number of sensors beyond five, provides 
little to no added value. We saw that while there may be a statistical significance, the 
additional sensors are of no practical value. However, there are a couple of critical 
junctures. For example, the addition of a single sensor provides a noticeable percent 
decrease in the red score, suggesting that even a relatively small rise in information 
superiority can have an immediate impact. Adding an additional sensor—bringing the 
total to two—again provides a recognizable increase to blue performance. The addition 
of three and four sensors appears to have little impact, with the next jump in performance 
occurring at five sensors and going no higher. To some degree, this is counterintuitive. 
Given a situation where blue has nine sensors and every location covered, we would tend 
to believe that this scenario would dominate all others. This is not the case, however, and 
it poses some interesting questions to those who are interested in developing persistent 
intelligence, surveillance, and reconnaissance (ISR) systems and the role they should 
play. While the work required by blue in executing its decision was not specifically 


measured, it should be clear that an increased workload was applied to the blue force 
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when there were nine sensors (vice only five) due to the increased calculation required. 
However, there was little to no measurable resulting improvement. This indicates 
another potential area of cost savings in terms of the tax placed on the decision maker, 


and reinforces the importance of identifying the proper scope for information superiority. 


Finally, the data reveal no critical points in terms of information quality, but 
rather, the slope of the decrease in performance as a result of increasingly poor 
information remains steady. The data suggest a slight mitigation of poor information by 
the increase in the number of sensors, but probably not to the degree sufficient to justify 
the addition of sensors. Additionally, while the data show that the gains of information 
superiority far outweigh the risks, there are, nonetheless, risks that need to be considered. 
No differences in risk or gain are noted to any noticeable degree based on the number of 


sensors. 
B. RECOMMENDATIONS 


There are a number of ways in which to extend this research into even more 


profitable studies. We will discuss a few of them here. 


First, without changing the current model there is room for additional exploration 
by adjusting the current design of experiment. In this experiment, a large proportion of 
the design points were used, ensuring that all possible combinations of red units and blue 
units were considered. This was at the expense of considering an increasing number of 
variations of sensor number, probability of time delay, and accuracy variation. Recall 
that there were 948 combinations of red and blue and an NOLH design that totaled 99 
rows. This represents somewhat of a disparity, particularly if one is interested more 
keenly in the impact of information quality. To do this, we recommend that red-blue 
combinations be limited to those plateau regions identified in the contour plots and an 
increased number of design points be created for the other three parameters. Not only 
would this provide more substance to the findings of this study, but it would also allow 
for a more detailed analysis of the risk versus gain issue. A potential question to be 
answered is, “How sensitive are risk and gain to information quality?” In other words, is 
risk or gain substantially increased or decreased at some level of information quality? 
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Furthermore, information superiority could be explored from the red perspective, where 
red receives the information advantage. Consideration could even be given to the 
circumstance where both sides are provided varying levels of sensors at the same time. 


All of these would provide added value to this research. 


There are also a number of ways to extend the model itself. Worthy of 
exploration would be the examination of varying the scoring system at the objectives. 
This could be accomplished by changing the basic game to some variation of a Blotto 
game, for example. A Blotto game is a type of two-person zero sum game that can still 
be solved using linear programming techniques. An advantage to Blotto games is that 
they have been broadly studied, thus providing additional background to the research. 
However, major adjustments would have to be made to the current implementation of the 
model to take into account the different optimal strategies that would be employed by 
both red and blue. Additionally, it is highly likely that the decision algorithm employed 
by blue would appreciably increase in complexity. Identifying an appropriate decision 
algorithm, even in a relatively simple model such as the one represented here, can be a 
challenging task. However, the value of the study would be in examining the value of 
information superiority over a broad range of decision-making scenarios and looking for 


broader trends in the value of information superiority to decision making. 


A variety of other complexities could be incorporated that would also require 
extensive model development. Consideration could be given to implementing types of 
information with varying degrees of importance. This would require a potential 
weighting scheme with each of these types of information. Additionally, the 
incorporation of non-homogenous red and blue forces goes hand in hand with adding 
types of information. A simpler means to increase complexity would be to expand the 
size of the gameboard and potentially adjust the current movement restrictions on both 
red and blue. Adjusting the levels of movement restrictions is akin to varying the level of 
flexibility inherent to the decision maker. The relationship between flexibility and the 
value of information superiority would be an interesting area of study itself. 


Furthermore, implementing a method for different types of sensors with probabilities of 
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detection that differ by type of sensor would represent a huge leap in complexity. This 
would be like a weapons-target assignment problem but with sensors, where questions of 


sensor mixes and sensitivity to probability of detection could be addressed. 


These suggestions represent a slice of the many ways this research can be 
improved and extended. Game theory is the common thread and represents a rich and 
robust subject area that has a strong history of addressing the operational issues facing 
today’s decision makers. While not revolutionizing the study of information superiority, 
this research demonstrated that game theory can be effectively applied in garnering 
insights into a critical issue facing the military operations research community. As a 
community we must be able to effectively tackle the role and value of information on the 
battlefield. Future research should continue to exploit game theory concepts and 


applications to further explore the questions posed by this study and related questions. 
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